Time delay and integration (TDI) is an imaging technique that uses a two-dimensional array image sensor to capture images from an imaging platform that is moving relative to the imaged object or scene. As the object or scene moves across the array, the image sensor takes multiple samples and sums these samples in order to improve the signal to noise ratio as compared to a single frame capture of the image sensor. This improvement to signal to noise ratio makes TDI imaging techniques particularly well-suited to applications with low light levels or fast moving objects. Example applications can include medical imaging, machine vision, roll or conveyor belt inspection systems, or terrestrial imaging from aircraft or satellites.
Conventionally, charge-coupled device (CCD) technology has been used for TDI applications because CCDs intrinsically operate by shifting charge from pixel to pixel across the image sensor. This shifting of charge allows the CCD image sensor to accomplish the integration (or adding) of the multiple samples without the addition of complex circuitry and accompanying noise. However, CCD technology is relatively expensive to fabricate and CCD imaging devices consume much more power than comparably sized devices implemented using complementary metal-oxide semiconductor (CMOS) technology.
It could be advantageous to combine CCD and CMOS technologies to generate a hybrid device in which the light capture and associated charge transfer to achieve TDI operation are accomplished with a CCD-like structure that is implemented on a CMOS substrate. By virtue of the CMOS substrate, signal processing and I/O could be implemented with CMOS circuitry in order optimize circuit integration, noise performance, power dissipation, speed, and size. However commercial silicon fabrication technologies that would allow such hybridization are not readily available at present.
Another option could be to hybridize with physically different chips. For example the light capture and TDI charge transfer could be accomplished on a CCD substrate, and the signals from that CCD chip routed to a carrier CMOS chip on which signal processing and final I/O could be implemented. The technology to develop both chips separately is readily available, but the cost associated with a two chip solution plus the expense of chip interfacing (through silicon vias, bond wires, etc.) is still a hurdle.
An alternative to CCDs is to use a CMOS pixel and to perform the summing operation in the voltage domain outside the CMOS pixel array. The signal is converted to voltage directly inside the CMOS pixels and requires adders outside the pixel array. These implementations can be accomplished on a single chip with all-CMOS circuitry. All-CMOS solutions can allow for lower power, higher degree of integration, and higher speed relative to all-CCD implementations, but existing designs suffer from noise issues, such as geometric distortion and a degraded modulation transfer function in the along track direction, when compared to existing CCD TDI image sensors.
Many CMOS sensor arrays use a 4-transistor (4T) pixel structure to maximize fill factor and attempt to minimize noise. The 4T pixels is clocked using a “rolling shutter” technique. This pixel architecture and method of clocking allows for true correlated double sampling (CDS) which largely removes the dominant KBTC reset noise. The use of rolling shutter clocking can causes artifacts in the acquired image since not all pixels are integrating over the same time period. Using a rolling shutter operation to achieve TDI functionality without any compensatory circuitry or timing will result in the severe loss of responsivity and/or severe modulation transfer function (MTF) degradation in the scan direction.
Attempts at performing TDI in non-CCD image sensors are described in U.S. Pat. No. 6,906,749, EP 1667 428 A2 and U.S. Pat. No. 5,828,408. A CMOS TDI sensor implementation having active pixels with snapshot shutter capability has been described by Pain et al. (“CMOS Image Sensors Capable of Time-Delayed Integration,” NASA Tech Brief Vol. 25, No. 4). The snapshot shutter capability means that all pixels start and stop their integration period simultaneously. As a scene pixel moves its focus from pixel to pixel along the column of the CMOS image sensor array, the signal from this scene pixel is multiply sampled, and each sample is integrated onto the storage capacitor on one integrator in the integrator array. The TDI imager must continually keep track of which pixel's output is added to which integrator because the scene pixel moves from pixel to pixel in the imaging array. After a given scene pixel has moved through all rows, the output of the corresponding integrator is sent to an analog-to-digital converter, and the integrator is reset so that it can begin the integration of the new scene pixel that moves into the field of view of the CMOS image sensor array. Of course, all pixels in the column must be connected in turn to the appropriate integrators in the time it takes for a ground pixel to move from one imager pixel to the next. The signal is dumped in a snapshot mode that eliminates motion artifacts that would otherwise be caused by the fact that each imager pixel is addressed at a slightly different time in a rolling shutter operation. The drawback of this approach is that an additional array per column of sample and hold capacitors are required to hold the pixel reset level per pixel. These memory elements are large compared to the pixels themselves and therefore have a significant impact on die size. Another drawback is that the signal must be stored on a node in the pixel for an appreciable fraction of the line time—this requires very careful shielding of the storage node from incoming light (low shutter leakage) or else spatial resolution in the scan direction will be compromised.
The performance challenges associated with developing high quality global shutter pixels, plus the requirement for additional sample and hold circuitry, can be eliminated by using rolling shutter 4T or 5T pixels. In rolling shutter pixels the read operation to eliminate kBTC reset noise can be accomplished without the use of additional sample and hold circuitry. Shutter leakage requirements can also be relaxed because charge is not stored on the pixel sensor node for any appreciable duration. However architectures based on rolling shutter pixels have other shortcomings that must be addressed. In particular, rolling shutter operation causes artifacts in the acquired image because not all pixels are integrating over the same time period. Each pixel must have acquired exactly the same portion of the scene; otherwise, the modulation transfer function (MTF) in the scan direction will be degraded. If the pixel integration time matches the travel time of the scene from one pixel to the next one, then all pixels must be operated synchronously. This cannot be done with rolling-shutter active CMOS pixels as they typically share the same column bus and only one row can be read at the a time.
LePage et al., “Time-Delay-Integration Architecture in CMOS Sensors” proposes a TDI architecture based on a rolling shutter pixel in which pixel position and pixel clocking is modified in order to compensate for the rolling shutter artifacts. In the described architecture adjacent columns are successively offset by 1/N of a pixel, where N is the number of pixel rows in the scan direction. Two approaches are described, one in which the readout is performed on entire individual columns at one time, and one in which pixels are read out in a diagonal fashion. In the first architecture the number of amplifiers that work in parallel is equal to the number of rows, N, in the scan direction, i.e. one amplifier per row. Since the number of columns in TDI image sensors is typically much larger than the number of rows, the readout speed of this architecture is severely limited relative to more conventional architectures in which all columns can be processed in parallel, i.e. one amplifier per column. The second architecture resolves this issue by using a column-wise readout architecture, but requires that redundant amplifiers be added. Specifically, for N rows in the scan direction, this architecture requires (N+1)/N times as many amplifiers as would ordinarily be required. There are at least three negative impacts associated with these redundant amplifiers: 1) N+1 amplifiers must be squeezed into the space of N columns which increases the size of the smallest pixel pitch that could otherwise be achieved; 2) manufacturing yield is negatively impacted, and; 3) (N+1)/N times as much power is required to run the amplifiers. Another issue is the length of the data bus lines in the pixel array are ˜50% longer than normal because these lines must run diagonally. This increases the drive load that the pixel amplifiers see (impact on speed and/or power dissipation), decreases yield, decreases pixel fill factor, increases opportunity for signal crosstalk, and may constrain the smallest pixel pitch that can be achieved with any given CMOS fabrication technology. The diagonal architecture also limits flexibility in how the pixel array is used. For example, the way it breaks spatial symmetry makes it unsuitable to be reconfigured via simple changes in row read timing to be operated as a multichannel device in order to perform colour linescan or hyperspectral image sensing.